Musical Acoustics and Speech Communication: Musical Pitch Tracking and Sound Source Separation Leading to Automatic Music Transcription II

This paper describes research aimed at building ‘‘active music listening interfaces’’ to demonstrate the importance of music understanding technologies, including sound source separation and F0 estimation, and the benefit they offer to end users. Active music listening is a way of listening to music through active interactions. Given polyphonic sound mixtures taken from available music recordings, our active music listening interfaces enrich end-users’ music listening experiences. For example, by suppressing drum sounds and adding other drum sounds, Drumix Yoshii et al., IPSJ Journal, 48 3 , 2007 enables a user to change the volume and timbre of drum sounds separated from the mixtures and rearrange rhythmic patterns of these drum sounds during playback. In addition, during the playback of a song, LyricSynchronizer Fujihara et al., Proc. of IEEE ISM 2006 displays scrolling lyrics and highlights the phrase currently sung. Because lyrics are automatically synchronized with vocal melodies separated from polyphonic sound mixtures using our predominant-F0 estimation method PreFEst , a user can easily follow the current playback position and click on a word in the lyrics to listen to it. These interfaces can also be regarded as ‘‘augmented music-understanding interfaces’’ that facilitate deeper understanding of music by end users.